Interaction Driven Quantum Phase Transitions in Fractional Topological Insulators 
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We study two species of (or spin- 1/2) fermions with short-range intra-species repulsion in the 
presence of opposite (effective) magnetic field, each at Landau level filling factor 1/3. In the absence 
of inter-species interaction, the ground state is simply two copies of the 1/3 Laughlin state, with 
opposite chirality, representing the fractional topological insulator (FTI) phase. We show this 
phase is stable against moderate inter-species interactions. However strong enough inter-species 
repulsion leads to phase separation, while strong enough inter-species attraction drives the system 
into a superfiuid phase. We obtain the phase diagram through exact diagonalization calculations. 
The FTI-superfiuid phase transition is shown to be in the (2-|-l)D XY universality class, using an 
appropriate Chern-Simons-Ginsburg-Landau effective field theory. 

PACS numbers: 



I. INTRODUCTION 

Topological phases of matter are of strong interest to 
physicists^. Exciting recent developments in this area 
are the discoveries of topological insulators, which can be 
viewed as time-reversal invariant analogs of integer quan- 
tum Hall (IQH) states^. A natural question is if the na- 
ture supports time- reversal invariant analogs oi fractional 
quantum Hall (FQH) states, namely fractional topolog- 
ical insulators (FTIs). Just like FQH states, FTIs are 
expected to be stabilized by strong interaction. 

Perhaps the simplest FTI stated is two (decoupled) 
copies of Laughlin states for up- and down-spin electrons, 
with opposite chirality. Such states are the exact ground 
states of model Hamiltonians"* with special short-range 
repulsions between electrons with the same spin, but no 
interaction between electrons with opposite spins. More 
recently lattice models that support FTIs have been con- 
structed in both two dimension (2D)^ and three dimen- 
sion (3D)*^. Since FTIs are stabilized by specific forms of 
interactions, it is important to understand the stability 
of FTIs when interactions are varied, and in particular, 
what kind of competing phases FTIs yield to when quan- 
tum phase transitions (QPTs) are triggered by varying 
interactions. Also of strong interest is the nature of such 
QPTs, and the critical properties of the QPT when it is 
continuous. 

Motivated by the above we consider the stability of 
the simplest FTI stated in the presence of more general 
electron-electron interaction, in particular inter-spin in- 
teractions. We show that sufficiently strong inter-spin 
repulsion leads to phase separation, while inter-spin at- 
traction drives the system into a superfiuid (SF) phase. 
The FTLSF transition is 2nd order, and in the 3D XY 
universality class. We also show that the FTI and SF 
phases have natural analogs in bilayer quantum Hall sys- 
tems at total filling factor 1 with layer imbalance^; our 
results are thus relevant to that system which is of strong 
interest in its own right. 



The rest of the paper is organized as follows. In Sec.HIl 
we introduce the simplified model and discuss the re- 
lated symmetry. In Sec. IIIII we focus on the numerical 
result on various properties of different phases, including 
energy spectrum, pair correlation function, and global 
phase diagram. The corresponding effective field theory 
is constructed in Sec. IIVI Finally we discuss the essen- 
tial relation between SF phase and bilayer layer quantum 
Hall systems at total filling factor 1 in Sec. |Vl 



II. MODEL AND SYMMETRY 

In this work we use a simple model to study FTIs, in 
which Landau levels are created by the spin-orbit cou- 
pling in the presence of a strain^ii^. Unlike external mag- 
netic field, strain does not break time reversal symmetry. 
As a result electrons with opposite spins experience op- 
posite effective orbital magnetic fields, and Landau lev- 
els with opposite chirality appear for opposite spin ori- 
entations. The single-particle Hamiltonian is given by 
■^□(r) = 2^(p + f A'^)^ with spin dependent vector po- 
tential A'^ = o'i3off(j/, 0, 0), with a = ±1 for up- and 
down-spin electrons. Ref. y! discussed how to realize such 
a situation in specific semiconductor materials; here we 
use this as an idealized model to study the stability of 
FTIs. 

Just like in FQH states electrons are confined to the 
lowest Landau level, and electron-electron interaction 
dominates. In this work, we consider hard-core inter- 
action Vil^V'^ S'^'^^v) between same spin fermions, and 
Vb/|(5'^''(r) between opposite spin fermions; Ib is mag- 
netic length. The intraspin interaction Vi is the energy 
of a pair of electrons with relative angular momentum 1 
which is first introduced by Haldane^. It captures the 
essence of the topological phases and gives rise to the 
exact model ground state, Laughlin state. The interact- 
ing Hamiltonian 'Hint is defined by setting Vi = V siTi{ip) 
and Vo = V cos((^), where the parameter ip tunes the rela- 
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tive strength between the Vi and Vq terms. Throughout 
the paper we set the overah energy scale V as energy 
unit and magnetic length /b as length unit. We also use 
torus geometry by imposing (magnetic) periodic bound- 
ary conditions. In second quantization the interacting 
Hamiltonian reads 



(1) 



where cj^ is the creation operator for an electron with 
spin index a and Landau orbital index j. 

The full symmetry analysis of FQH systems at ratio- 
nal filling factors was provided by Haldane^. Here we 
generalize his analysis, and show eigen states of our sys- 
tem can be characterized by a 2D wave vector K for 
arbitrary filling factor. To prove this we introduce mag- 
netic translation operator for particle j with spin a: 
f/(r) = exp{ir-nj/;i}, where = + fA'"(rj). 

f^{r) commutes with H^irj), but f/(a)f/(b) = 

exp{— icrz • (a x b)} (b)2y (a) and thus do not com- 
mute with each other in general, due to the Berry phase 
induced by magnetic field. We now introduce the center- 
of-mass translation operator: r(r) = Y\ja'^j ^^)- 
Tx and TV be center of mass translations by some arbi- 
trary amounts along x and y directions. Both Tx and 
Ty commute with the many body Hamiltonian % which 
can be separated into a center of mass term and relative 
motion terms. Since opposite orientation spins experi- 
ence opposite orbital magnetic fields in the present sys- 
tem, the Berry phases picked up by up- and down-spin 
electrons cancel when they have the same numbers, as 
a resuh of which TxTy = TyTx- Thus the Hamilto- 
nian can be simultaneously diagonalized along with Tx 
and Ty, with eigen states labeled by a 2D momentum 
quantum number. 



III. PHASE DIAGRAM 

The simplest FTI states, in our language, corresponds 
to the ground states at (/Jp = 7r/2, in which case Vb = 
and thus the two spin species decouple, and there are 3 x 
3 = 9 exactly degenerate ground states that are separated 
from excited states by a finite gap. Such degeneracy is 
a topological property of the phase, and robust against 
small perturbations^. 

As shown in FIG.[l](a), when we turn on a small Vq by 
having = 0.47r, we still have 9 nearly degenerate low 
lying states that are well separated from excited states 
by a gap; this clearly indicates the system is still in the 
FTI phase. 

However sufficiently strong Vq destabilizes the FTI 
phase. As shown in FIG. [1] (b), at — O.OItt, where 
Vb ^ 14 > 0, there are many low-lying states with 
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FIG. 1: (Color online) Representative low-energy excitation 
spectra, obtained by exact diagonalization studies on a torus 
aX v-f = = 1/?) with number of flux quanta — 15, 
and A'^ — = 5, mostly based on square geometry with 
magnetic periodic boundary conditions. The states are la- 
beled by the magnitude of the conserved wavevector |K| in 
units of I \J L^Ly. (a) The fractional topological insulator 
at = 0.47r. Wave vectors of 9 nearly degenerate ground 
states are marked. They are separated from excited states 
by a clear gap. (b) Strong inter-spin repulsion at iy9 = O.OItt 
that leads to phase separation between up- and down-spin 
electrons, (c) Strong inter-spin attraction at = O.OQtt that 
stabilizes a paired superfluid phase. The unique ground state 
is at K = 0, and low-energy excitations form a Goldstone 
mode with linear dispersion. Data from systems with the 
same size but different rectangular geometries are included in 
(c) to obtain more K's in order to reveal the linear dispersion 
more clearly. 



no gap. In this case we expect the up- and down- 
spin electrons phase separate into v^^ — 2/3, i^j, — 0, 
and — Ij'i^v^^ — regions. This is supported 
by the opposite-spin pair correlation function g^^(v) = 
L,Ly/iN^N%){GS\ -^(''(r + r| - rj)|GS), as iUus- 
trated in Fig. [2] (a): It goes to zero at origin r = 0, while 
shows a large hump around y/Ly = 0.5, the farthest pos- 
sible place in our finite size system (with periodic bound- 
ary condition). The low- lying states correspond to fluc- 
tuations of the boundary between the separated regions, 
which do not have clear patterns in their quantum num- 
bers. 

The situation is more interesting when we switch on at- 
tractive inter-species interaction, which can lead to pair- 
ing between opposite spin electrons. Such pairs see zero 
magnetic field, and can thus condense; their condensa- 
tion leads to a superfluid (SF) phase with a sponta- 
neously broken U(l) symmetry. This is what happens 



3 



(a) <p = O.Olit 




y/Lv' 0.25 
' 



(b) 


p = 0.997U 




(r)/ 




3 




2.5 




2 




1.5 




1 

0.5 




0.5 \, 




0.25 " 











To determine the phase diagram of the system and 
the precise locations of the phase boundaries, we calcu- 
late overlaps of the actual ground state with ideal model 
states for the FTI and SF phases. The former is sim- 
ply the direct product of two decoupled Laughlin states 
obtained at ip-p — 7r/2, and the latter is chosen for the 
ground state at = tt, whose explicit wave function is 
Eq. (2) of Ref. llj (more on this later). The results are 
presented in Fig. [3l from which it is clear that phase sep- 
aration (PS) is a first-order instability of the FTI phase, 
while the FTI-SF transition appears to be continuous. 



FIG. 2: (Color online) Two-particle correlation function 
g^'^(r) between diflerent spin orientations, obtained by ex- 
act diagonalization on a torus with square geometry Lx ~ Ly 
for number of electrons A*'^ = A'^"'' = 5 and number of flux 
quanta Nf = 15, as a function of {x/ Lx,y /Ly). (a) The large 
hump around y/Ly = 0.5 with K = (5, 7) indicates the phase 
separation^'' . (b) The maximum of g^^ at the origin is indica- 
tive of the paired character of the ground state with K = 
which has highest symmetry in the Brillouin zone. 



(a) 



(b) 1 



0.8 



0.6 



0.4 



0.2 



PS 




FIG. 3: (Color online) Overlaps between actual ground state 
with ideal model states for the fractional topological insulator 
{ipF — 7r/2) and superfluid {(ps = i") phases (b), and resultant 
phase diagram and boundaries (a). FTI, SF and PS stand 
for fractional topological insulator, superfluid and phase sep- 
aration respectively. These overlaps are obtained by exact 
diagonalization on a torus with square geometry Lx = Ly for 
number of electrons A*'^ — N'^ — 5 and number of flux quanta 
Nf — 15. The inset presents results with system size Nf = 12 
(with A''^ = A''-'' — 4), showing very little size dependence. 



a,t (f = O.OQtt, where —Vq ^ Vi > 0. The pairing char- 
acter of ground state is clearly manifested in the pair 
correlation function g"^-^ (r) in Fig. [5] (b) , which sharply 
peaks at origin. Stronger evidence for the SF nature of 
the phase is in the spectrum of Fig.[l](c), which features a 
non- degenerate ground state at K = 0, and a low-energy 
mode with linear dispersion at long wavelength. This 
linear mode is the SF Goldstone mode. 



IV. EFFECTIVE FIELD THEORY 

In this section we develop an effective field theory that 
describes the FTI and SF phases on equal footing, and in 
particular, the phase transition between them. Our start- 
ing point is the Chern-Simons-Landau-Ginzburg (CSLG) 
theorjsi^ for FQH states, in which the Laughlin state is 
viewed as a superfiuid state of composite bosons made of 
electrons bound to appropriate amount of fiux. Since the 
FTI can be viewed as two copies of the Laughlin state 
with opposite chirality, it is thus natural to attach oppo- 
site flux to electrons with opposite spins. This leads to 
the following effective field theory lagrangian density: 

L = + + Lp + Lcs + ff(0V'tV'i + </'^t^4.)' 
io- = '(/'^(i9t - ao)V^cr - mIV'ctP 

(3) 

f + (4) 
aldi^al - a-ld^a\ .(5) 



— |(-zV-A'^-a-)^.p 



+ |(-iV - a''" - a^)( 



Here ip^ are composite boson fields for (bosonized) elec- 
trons, while 3°^ is the Chern- Simons (CS) gauge field that 
attaches flux to spin-cr composite bosons, which cancels 
extenal gauge field — — A-*- in average. 6 deter- 
mines the amount of flux attached to each particle; in 
the present case 9 = 3tt for the 1/3 Laughlin state. No- 
tice the minus sign between Lj^ and L^^; this indicates 
the flux is in opposite directions for up- and down-spin 
composite bosons. The presence of pairing (or attrac- 
tive) interaction between up- and down-spin electrons is 
encoded by introducing a pair fleld 0, with lagrangian 
density Lp. It represents bound state of up- and down- 
spin composite bosons, and sees no external gauge fleld 
A, due to the cancelation between A''^ and A^. On the 
other hand it couples equally to the CS gauge fields a^ 
and a.^. Using terminology familiar in cold atom con- 
texts, we call the parameter 6 detuning; positive 6 favors 
unbound electrons while negative S favors pair forma- 
tion. Thus decreasing 6 corresponds to increasing pairing 
strength. The g term in Eq. ^ describes pair forma- 
tion and decay processes. Generic density-density inter- 
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actions among the particles are kept implicit and repre- 
sented by • • • . Due to the relation between particle and 
CS flux density enforced by the CS terms, such density- 
density interactions can also be written in terms of CS 
flux density 6°" — e^^dta'^ . 

The distinction between the FTI and SF phases is 
the following. In the FTI phase, which is topologically 
equivalent to two independent FQH states, composite 
bosons with up- and down-spins both condense, namely 
(ipcr) 7^ Oi^; this gives rise to Anderson-Higgs masses 
to the CS gauge fields, and the corresponding Meisnner 
effect is equivalent to incompressibility responsible for 
FQHE. Due to the g term in Eq. this implies (0) ^ 0. 
In the SF phase however, there is only a pairing gap that 
penalizes imbalance between up- and down-spin electron 
numbers, but no overall charge incompressibility. This 
suggests in the SF phase {ipa} = while we still have 
{4>) 7^ 0, as couples only to the combination 
whose flux is the difference between up- and down-spin 
electron density. Thus the FTI and SF phases differ by 
one U(l) condensate, and the transition is driven by the 
appearance/disappearance of this condensate. The situ- 
ation is somewhat similar to a transition between inte- 
ger and fractional quantum Hall states driven by pair- 
ing interaction studied in Ref. ITsl . In particular, if we 
completely suppress the fluctuations of CS gauge fields, 
the critical theory will reduce to that of the (2+l)D XY 
model just as in Ref. [l^ with the critical U(l) field being 

Also like in Ref. [Isl, fluctuations of CS gauge fields 
changes the physics, but in a very different manner here. 
To proceed, we introduce new combinations of CS gauge 
fields: = (aj^ ± a^)/2, in terms of which Eqs. (|3I4I5P 
take the form 

+ T^I(-*V - A - a+ - a-)V;tl' + • • • , (6) 

Li = ip^{idt-a+ +a^)i/ji- fi\tpi\'^ 

+ ^l(-»V + A-a++a-)V'^|2 + ... , (7) 

Lp = ^iidt~2a+)(j)-{2fi-SM^ 

+ ^|HV-2a+)0p + ... , (8) 

Lcs = ie^^"a+9.a^. (9) 

Notice in particular the CS term is a bilinear coupling 
between a+ and a~ , of the BF formi^. Noting that since 
{(j>) 7^ 0, and thus a+ acquires an Anderson-Higgs mass 
of the form Aa+^a^ throughout the phase diagram, we 
can safely integrate out; due to their coupling to aj 
through Lcs I this will generate Maxwell terms of the form 
F^'^^ Fq- — e^^ej . Integrating over a J enforces the pro- 
portionality constraint between — F,^ — eijdioj and 
total density; this allows us to write some of the density- 
density interactions in "• • • " in the form of the remaining 



Maxwell terms F = |6 p. Thus the effective ac- 

tion of a~ after integrating out a+ is that of Maxwell 
theory F~^'^F~j^, with space-time re-scaled according to 
the non-universal photon velocity, which is the Goldstone 
mode (or sound) velocity in the superfiuid phase. 

As a result the final critical theory for the FTI-SF 
transition is that of Ginzburg-Landau-Maxwell theory in 
(2-t-l)D, with Euclidean lagrangian density (after proper 
re-scaling of the critical field ip~), which is dual to the 
(2-|-l)D XY modeli^: 

Leff[^_,a-] = |(a^-za;)^_p-A|V_p + t7|V'_|4 

+ aF-^'^F-^ + ■■■ . (10) 

Here "• • • " represent irrelevant terms involving higher or- 
der of derivatives or powers of fields (either in the orig- 
inal lagrangian or generated by integrating out a"*"). In 
the SF phase, (ip-) — 0, the a~ gauge field is in the 
Coulomb phase which supports a single branch gapless 
photon mode with linear dispersion; this corresponds to 
the Goldstone mode of the SF phase. In the FTI phase 
{ip^) ^ (as a result we have two condensates, including 
{ip+) ^ 0), and the a~ gauge field is in the Higgs phase 
where there is no gapless excitation. The transition is in 
the (2-1- 1)D XY universality class due to the duality. 



V. RELATION WITH BILAYER QUANTUM 
HALL SYSMTEM AT TOTAL FILLING FACTOR 

^■t = 1 

Our model has close relation with bilayer quantum Hall 
system at total filling factor i/t = i^top + ^bottom = 1 
(where vtop and vj^ottom correspond to filling factors of 
top and bottom layers respectively), especially in the 
presence of layer imbalance. This relation is revealed 
by making a particle-hole transformatiortii for the lower 
layer and describe it in terms of holes, which results in 

hole filling factor Vhole = 1 - J^bottom = l^top = V^lectron- 

Since the holes carry opposite charge as the electrons, 
the chirality due to external magnetic field is opposite for 
them as well. The ground state in the strong particle- 
hole pairing limit is nothing but the Halperin 111 statei^ 
written in terms of electron coordinates, which is the 
ideal model state for the pseudospin ferromagnetic or in- 
terlayer exciton condensate stateiS.. The corresponding 
wave function written in terms of electrons in top layer 
and holes in bottom layer is presented in Ref. [III. The 
specific case of = = \/i correspond to vtop = 1/3 
and Vbottom = 2/3, and the FTI and SF states discussed 
here correspond to decoupled single layer FQH and in- 
terlayer exciton condensate states. A transition between 
them has been observed experimentally^, and the theory 
developed here for the FTI-SF transition applies to that 
case as well. We note while bilayer quantum Hall liq- 
uid is a fascinating system in its own right, it may now 
also be used as testing ground for some of the theoretical 
predictions^^ and proposed techniques-^ to probe FTIs 



5 



due to their close relation, given the fact that there is no 
candidate system for FTI at this point. 
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